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DETAILED ACTION 
Response to Amendment 

1 . The amendment was received on October 18, 2004. Claims 1-30 are pending. 

Response to Arguments 

2. Applicant's arguments filed 10/18/2004 on page 14,lines 24-26 and page 16, 
lines 4-6 have been fully considered but they are not persuasive. 

Page 14, lines 24-26 states, "Shum does not disclose or suggest applying a 
transform to each image in a composite image to change a composite image having a 
first center of projection to a composite] image having a second center of projection." 

However, Shum does disclose applying a transform ("Hough transform" in col. 7, 
line 27.) to each image (Fig. 3, numerals 302a-n) in a composite image (Fig. 3, num. 300 
is H a set of concentric... mosaics (col. 2, lines 58,59) [which corresponds to the claimed 
composite image because each mosaic is combined concentrically to form an overall 
composite image or mosaic].") to change a composite image (Fig. 3, numerals 302c and 
302b are mosaics that are locally warped together in col. 1 1 , lines 27-31 . Where 302c is 
warped into 302b because of the direction of viewing as shown by the arrow 306 of fig. 

3. ) having a first center of projection (Fig. 3,num. 302b is a first center of projection or 
"new image" in col. 1 1 , line 29 with respect to image 302c.) to a composite] image (Fig. 
3, num. 302b and 302a) having a second center of projection (Fig. 3, num. 302b is a 
second center of projection or "new image" in col. 1 1 , line 29 with respect to image 
302a.)." Note that a direction of viewing as shown in fig. 3, num. 306 determines which 
image is a new image with respect to another image. 
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3. Applicant's arguments filed 10/18/2004 on page 15, lines 19-21 have been fully 
considered but they are not persuasive. 

Page 15, lines 19-21 states, "Shum, however, does not transform a composite 
image that has been corrected for perspective distortion about a first center of projection 
to create a second composite image corrected for perspective distortion about a 
different center of projection." 

However, Shum does transform (A "Hough transform" in col. 7, line 27 is used for 
"warping... mosaics (col. 11, lines 16,17).") a composite image (Fig. 3, num. 302c and 
302b are mosaics that are locally warped together in col. 1 1 , lines 27-31 .) that has been 
corrected for perspective distortion (The warping transform "preserves structural 
features (col. 10, lines 12,13)." Note that structural features can include features related 
to perspective such as "edges" in col. 10, line 8.) about a first center of projection (Fig. 
3,num. 302b is a first center of projection or "new image" in col. 11, line 29 with respect 
to image 302c.) to create a second composite image (Fig. 3, num. 302b and 302a is a 
second composite image where 302b is a new image or created image, using the 
warping transform, with respect to num. 302a.) corrected for perspective distortion (The 
warping transform "preserves structural features (col. 10, lines 12,13)." Note that 
structural features can include features related to perspective such as "edges" in col. 
10, line 8.) about a different center of projection (Fig. 3,num. 302b is a second center of 
projection or "new image" in col. 11, line 29 with respect to image 302a.). 
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4. Applicant's arguments on page 15,lines 23-26 with respect to claim 1 have been 
considered but are moot in view of the new ground(s) of rejection using Shum et al. (US 
Patent 6,643,413 B1). 

Claim Objections 

5. The following quotations of 37 CFR § 1 .75(a) is the basis of objection: 

(a) The specification must conclude with a claim particularly pointing out and distinctly claiming 
the subject matter which the applicant regards as his invention or discovery. 

6. Claim 1 5 is objected to under 37 CFR § 1 .75(a) as failing to particularly point out 
and distinctly claim the subject matter which the applicant regards as his invention or 
discovery. 

Claim 15, line 6:"the perspective" ought to be amended to "a perspective". 

Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

8. Claims 1-4,11-19 and 26-30 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Shum et al. (US Patent 6,643,413 B1). 

Regarding claims 1,15,16 and 30, Shum et al. discloses a computer- 
implemented image processing (fig. 1 shows a computer 20 with a medium 29.) 
method, comprising: 
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a) providing a composite image (fig. 3,num. 302b is a mosaic image or "new 
image" in col. 11, line 29.) derived (Fig. 3, num. 302b is a mosaic image or "new image" 
in col. 1 1 , line 29 derived from mosaic images 302c, 302b and 302a.) from a plurality of 
component images (fig. 3,num. 302c,302b and 302a are mosaics.) including a first 
image (fig. 3, num. 302c is a mosaic.) and a second image (fig. 3, num. 302b is a 
mosaic), the composite image (fig. 3,num. 302b is a mosaic image or "new image" in 
col. 1 1 , line 29.) including the first image (fig. 3, num. 302c is a mosaic.) as a center of 
projection (fig. 3,num. 302c is a mosaic image or "new image" in col. 1 1 , line 29.) and a 
modified version (The second image of fig. 3, num. 302b is modified or warped into the 
first image of fig. 3, num. 302c.) of the second image (fig. 3, num. 302b is a mosaic), the 
modified version of the second image (The second image of fig. 3, num. 302b is 
modified or warped into the first image of fig. 3, num. 302c.) having been corrected for 
perspective distortion (The warping transform "preserves structural features (col. 10, 
lines 12,13)." Note that structural features can include features related to perspective 
such as "edges" in col. 10, line 8.) relative to the first image (fig. 3, num. 302c is a 
mosaic); 



Application/Control Number: 09/941 ,427 Page 6 

Art Unit: 2621 

b) receiving a single user input including an instruction to change [the] a 
perspective (Fig. 1 1 ,num. 1 104: MOVEMENT OF VIRTUAL CAMERA) of the composite 
image (fig. 3, num. 302b is a mosaic image or "new image" in col. 11, line 29 and shown 
in fig. 1 1 , num. 1 1 02: CONSTRUCTION OF MANIFOLD MOSAICS.) to make the 
second image (fig. 3 f num. 302b is a mosaic.) the center of projection (fig. 3 f num. 302b 
is a mosaic image or "new image" in col. 1 1 , line 29 based upon the MOVEMENT OF 
VIRTUAL CAMERA and shown in fig. 3,num. 306.) ; and 

c) in response to the input (Fig. 1 1 .num. 1 104: MOVEMENT OF VIRTUAL 
CAMERA), automatically: 

c1) determining a transformation (A "Hough transform" in col. 7, line 27 
is "augmented" in col. 7, line 30. Thus a transform is augmented from another 
transform, so a transform is determined.) for mapping a set of reference points (fig. 5 
shows a set of reference points, 502-508, that correspond to the second modified 
image.) in the modified version of the second image (The second image of fig. 3, num. 
302b is modified or warped into the first image of fig. 3,num. 302c.) to a set of reference 
points (Fig. 4 shows rays 402-408 of "view points... [in] ray space" in col. 6, lines 42-50 
that correspond to the second image of fig. 3, num. 302.) in the second image (fig. 3, 
num. 302b); 
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c2) applying the transform (A "Hough transform" in col. 7, line 27) to 
each of the plurality of derived (Fig. 3, num. 302b is a mosaic image or "new image" in 
col. 11, line 29 derived or transformed via the Hough transform from mosaic images 
302c,302b and 302a.) component images (fig. 3,num. 302c,302b and 302a are 
mosaics.) in the composite image (fig. 3,num. 302b is a mosaic image or "new image" in 
col. 11, line 29.) to generate a plurality of transformed component images (In fig. 10, line 
"segments" in col. 11, line 18 shown in n ,r 0 and -x<[ are transformed component images 
that correspond to component images of fig. 3, num. 302c,302b and 302a, respectively.), 
each of the transformed component images ("segments" in col. 11, line 18) having the 
second image (fig. 3, num. 302b or line segment r 0 .) as their center of projection (Line 
segments n and have the second image of fig. 3, num. 302b or r 0 as their center of 
projection depending on which relative direction as shown as arrows on the ends of a 
line 1002 of fig. 10 is traveled or hopped as mentioned in col. 11, lines 15-17. If the 
direction or travel or hopping is from the line segment of -n to r 0 , the center of 
projection is r 0 , because -Pi is projected or "warp[ed]" in col. 11, line 16 to r 0 . A similar 
case is presented when hopping from n to r 0 . Thus, r 0 is the center of projection for a 
plurality of images.), each of the transformed component images ("segments" in col. 1 1 , 
line 18) being corrected for perspective distortion (The warping transform "preserves 
structural features (col. 10, lines 12,13)." Note that structural features can include 
features related to perspective such as "edges" in col. 10, line 8.) relative to the second 
image (fig. 3, num. 302b or line segment r 0 .); and 
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c3) merging (fig. 11, num. 1106: RENDERING OF IMAGE merges or 
"locally warps" in col. 13, line 43 mosaic images to create a "new image" in col. 13, line 
42.) the second image (fig. 3, num. 302b or line segment r 0 .) and the plurality of the 
transformed component images (In fig. 10, line "segments" in col. 11, line 18 shown in n 
,r 0 and -Pi are transformed component images that correspond to component images of 
fig. 3, num. 302c,302b and 302a, respectively.) corrected for perspective distortion (The 
warping transform "preserves structural features (col. 10, lines 12,13)." Note that 
structural features can include features related to perspective such as "edges" in col. 
10, line 8.) relative to the second image (fig. 3, num. 302b or line segment r 0 .) to form a 
second composite image (Fig. 3, num. 302b is a composite image from images 302a 
and 302b.) that has the second image (Fig. 3, num. 302b) as its center of projection 
(Line segments n and have the second image of fig. 3, num. 302b or r 0 as their 
center of projection based on a direction or hopping as shown as arrows on the ends of 
a line 1002 of fig. 10 as mentioned in col. 11, lines 15-17.). 

Claims 2,3,4,17,18 and 19 are rejected the same as claim 15. Thus, argument 
similar to that presented above for claim 15 is equally applicable to claims 2,3,4,17,18 
and 19. 

Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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10. Claims 5-10 and 20-25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Shum et al. (US Patent 6,643,413 B1) and in view of Heckbert 
(Master's Thesis from the IDS). 

Regarding claim 5, Shum et al. does not disclose the limitation of claim 5, but 
does suggest a correspondence of points between images in col. 1, lines 31-34 and 
performs a correspondence function with points in col. 7, lines 45,46. 

However, Heckburt does teach the limitation of claim 5 as suggested by Shum of: 

a) a second image and a modified version of the second image each 
include a perimeter as shown by the bold lines of figure 2.8 of Heckburt. 

b) a plurality of reference points in the second image and the modified 
version of the second image are vertices on the perimeters on the second image and 
the modified version of the second image. Heckburt states, "In an interactive image 
warper one might specify the four corners of source and destination quadrilaterals.. .and 
wish to warp one area to the other..." as mentioned in Heckbud, page 19, first sentence 
of the section "Inferring Projective Mappings". Figure 2.8 of Heckburt shows how the 
four vertices are transformed from case 1 to case 3. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify Shum et al.'s teaching of corresponding points with Heckburt's 
teaching of corresponding point, because Heckburt's corresponding points provides a 
predictable transformation between images. Therefore users in 3D modeling and image 
distortion software can use the mappings to provide predictable features (lines, 
equispaced points, and angles) that are persevered (Heckburt, page 11, Section 2.2). 
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Regarding claim 6, Shum et al. and Heckburt teaches the method of claim 5, 
wherein: 

a) the first image includes a plurality of pixels and has a perimeter that 
defines a set of vertices which was addressed in claim 5, and 

b) transforming the first image based on the transformation includes: 
b1 ) transforming the vertices of the first image (Using fig 2.8 of 

Heckburt, case three is shown where the vertices of the figure on the left side of figure 
2.8 is transformed to the right-hand figure of figure 2.8.); and 

b2) transforming the pixels of the first image based on the 
transformation of the vertices which was addressed in claim 5. 

Regarding claim 7, Shum et al. and Heckburt teaches the method of claim 6, 
wherein: 

the transformation is represented as a transformation matrix or (M sub. sd) is a 
forward mapping matrix as mentioned in Heckburt, page 19, section: "Inferring 
Projective Mappings", line 7. 

Regarding claim 8, Shum et al. and Heckburt teaches the method of claim 7, 
wherein: 

the transformation matrix is derived from the vertices of the modified version of 
the second image. Heckburt teaches that all points in one image (u sub. k, v sub. k) is 
mapped to other points (x sub. k, y sub. k) in a corresponding image using vertices 
(k=0,1 ,2,3) for each point. 



Application/Control Number: 09/941 ,427 Page 1 1 

Art Unit: 2621 

Claim 9 is rejected the same as claim 8. Thus, argument similar to that presented 
above for claim 8 is equally applicable to claim 9. 

Regarding claim 10, Shum et al. and Heckbud teaches the method of claim 9, 
wherein the transformation matrix, M, is given by the matrix equation shown in 
Heckburt, page 19. 

Claims 1 1 and 13 are rejected the same as claim 4. Thus, argument similar to 
that presented above for claim 4 is equally applicable to claims 1 1 and 13. 

Regarding claim 12 a reference point ("1") in the left-hand figure shown in 
Heckburt, fig. 2.8 is moved to another position as shown in the figure of 2.8 on the 
right-hand side is move relative to other reference points 0,2, and 3 to change the 
shape of the figure on the left side of figure 2.8 to the figure on the right-hand side of 
figure 2.8. 

Regarding claim 14, Shum et al. and Heckburt teaches the method of claim 1 , 
wherein: 

a) the instruction to shift perspective is received as a single user input (A 
user moves a virtual camera in Shum et al., fig. 2, num. 204 during the step of figure 

1 ,num. 1 104 of Shum et al. to change the current view to a new view as mentioned in 
Shum et al. user col. 13, lines 34,35; and 

b) the determining and transforming steps are automatically performed in 
response to the user input (A user is able to move and rotate the virtual camera that has 
a desired view of an object. The desired view is "then rendered" as mentioned in Shum 
et al., col. 5, lines 55-60. Thus, as the virtual camera's current view is moved a 
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corresponding image view is rendered using the method of figure 1 1 .). 

Claim 20 is rejected the same as claim 5. Thus, argument similar to that 
presented above for claim 5 is equally applicable to claim 20. 

Claim 21 is rejected the same as claim 6. Thus, argument similar to that 
presented above for claim 6 is equally applicable to claim 21 . 

Claim 22 is rejected the same as claim 7. Thus, argument similar to that 
presented above for claim 7 is equally applicable to claim 22. 

Claim 23 is rejected the same as claim 8. Thus, argument similar to that 
presented above for claim 8 is equally applicable to claim 23. 

Claim 24 is rejected the same as claim 9. Thus, argument similar to that 
presented above for claim 9 is equally applicable to claim 24. 

Claim 25 is rejected the same as claim 10. Thus, argument similar to that 
presented above for claim 10 is equally applicable to claim 25. 

Claim 26 is rejected the same as claim 1 1 . Thus, argument similar to that 
presented above for claim 1 1 is equally applicable to claim 26. 

Claim 27 is rejected the same as claim 12. Thus, argument similar to that 
presented above for claim 12 is equally applicable to claim 27. 

Claim 28 is rejected the same as claim 13. Thus, argument similar to that 
presented above for claim 13 is equally applicable to claim 28. 

Claim 29 are rejected the same as claim 14. Thus, argument similar to that 
presented above for claim 14 is equally applicable to claim 29. 
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Conclusion 

1 1 . THIS ACTION IS MADE FINAL Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

1 2. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dennis Rosario whose telephone number is 703-305- 
5431 . The examiner can normally be reached on 6-3. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on 703-308-5246. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Dennis Rosario 
Unit 2621 




